It is well established that minimally-invasive surgery (MIS) techniques offer significant health benefits over their analogous laparotomic (or “open”) counterparts. Among these benefits are reduced trauma, rapid recovery time, and shortened hospital stays, resulting in greatly reduced care needs and costs. However, because of limited visibility to certain internal organs, some surgical procedures are at present difficult to perform using MIS. With conventional technology, a surgeon operates through small incisions using special instruments while viewing internal anatomy and the operating field through a two-dimensional monitor. Operating below while seeing a separate image above can give rise to a number of problems. These include the issue of parallax, a spatial coordination problem, and a lack of depth perception. Thus, the surgeon bears a higher cognitive load when employing MIS techniques than with conventional open surgery because the surgeon has to work with a less natural hand-instrument-image coordination.
These problems may be exacerbated when the surgeon wishes to employ other modalities to view the procedure. A modality may be any method and/or technique for visually representing a scene. Such modalities, such as intraoperative laparoscopic ultrasound, would benefit the procedure by providing complementary information regarding the anatomy of the surgical site, and, in some cases, allowing the surgeon to see inside of an organ before making an incision or performing any other treatment and/or procedure. But employing more than one modality is often prohibitively difficult to use. This is particularly the case when the modalities are video streams displayed separately on separate monitors. Even if the different modalities are presented in a picture-in-picture or side-by-side arrangement on the same monitor, it would not be obvious to the surgeon, or any other viewer, how the anatomical features in each video stream correspond. This is so because, the spatial relationship between the areas of interest at the surgical site, for example, surface, tissue, organs, and/or other objects imaged by the different modalities, are not aligned to the same view perspectives. As such, the same areas of interest may be positioned and oriented differently between the different modalities. This is a particular problem for modalities like ultrasound, wherein anatomical features do not obviously correspond to the same feature in optical (or white-light) video.
The problems may be further exacerbated in that the surgical site is not static but dynamic, continually changing during the surgery. For example, in laparoscopic surgery, the organs in the abdomen continually move and reshape as the surgeon explores, cuts, stitches, removes and otherwise manipulates organs and tissues inside the body cavity. Even the amount of gas inside the body cavity (used to make space for the surgical instruments) changes during the surgery, and this affects the shape or position of everything within the surgical site. Therefore, if the views from the modalities are not continuous and immediate, they may not accurately and effectively depict the current state and/or conditions of the surgical site.
While there is current medical imaging technology that superimposes a video stream using one modality on an image dataset from another modality, the image dataset is static and, therefore, not continuous or immediate. As such, the image dataset, must be periodically updated based on the position of the subject, for example the patient, and/or anatomical or other features and/or landmarks. Periodically updating and/or modifying the image dataset may introduce undue latency in the system, which may be unacceptable from a medical procedure standpoint. The undue latency may cause the image being viewed on the display by the surgeon to be continually obsolete. Additionally, relying on the positions of the subject, and/or anatomical or other features and/or landmarks to update and/or modify the image being viewed, may cause the images from the different modalities to not only be obsolete but, also, non-synchronous when viewed.
Accordingly, there currently is no medical imaging technology directed to providing composite real-time dynamic imagery from multiple modalities using two or more video streams, wherein each video stream from each modality may provide a real-time view of the medical procedure site to provide a continuous and immediate view of the current state and condition of the medical procedure site. Also, there currently is no medical imaging technology directed to providing composite imagery from multiple modalities using two or more video streams, wherein each video stream may be dynamic in that each may be synchronized to the other, and not separately to the position of the subject, and/or anatomical or other features and/or landmarks. As such, there is currently no medical imaging technology that provides composite real-time, dynamic imagery of the medical procedure site from multiple modalities.
Therefore, there is a need for a system and method of providing composite real-time dynamic imagery of a medical procedure site from multiple medical modalities, which continuously and immediately depicts the current state and condition of the medical procedure site and does so synchronously with respect to each of the modalities and without undue latency.